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Stochastic maximum principle for infinite 
dimensional control systems 

Kai Du* Qingxin Meng* 

Abstract 

The general maximum principle is proved for an infinite dimensional 
controlled stochastic evolution system. The control is allowed to take val- 
ues in a nonconvex set and enter into both drift and diffusion terms. The 
operator-valued backward stochastic differential equation, which charac- 
terizes the second-order adjoint process, is understood via the concept 
of "generalized solution" proposed by Guatteri and Tessitore [S1CON 44 
(2006)]. 
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1 Introduction 

1.1 Problem formulation and basic assumptions 

In this paper we shall always indicate by H a real separable Hilbert space and 
by ('j')ir an d || • \\h its inner scalar product and norm, respectively. Denote 
by B(H) the Banach space of all bounded linear operators from H to itself 
endowed with the norm ||T||s(_ff) '■— sup{||Tx||// : \\x\\h = 1}- 

Let T, F, P) be a probability space with the filtration F = (J-t)t>o gen- 
erated by countable independent standard Wiener processes {W z ;i £ N} and 
augmented with all P-null sets of T . For simplicity, we write formally / • dW = 
SiGN fi dW" with a sequence / = (fiji £N). We denote by E^' the conditional 
expectation with respect to Tt-, and by V the predictable ex-field on x [0, 1]. 

In this paper, we study an infinite-dimensional optimal control problem gov- 
erned by the following abstract semilinear stochastic evolution equation (SEE) 

dx(t) = [Ax(t) + f(t, x(t),u(t))] dt + g(t, x(t), u(t)) ■ dW u 
z(0) = z , (1.1) 

where x(-) is the state process and u(-) is the control. The control set U is a 
nonempty Borel-measurable subset of a metric space whose metric is denoted by 
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dist(- , •). Fix an element (denoted by 0) in U, and then define \u\u = dist(u,0). 
An admissible control u(-) is a [/-valued predictable process such that 

sup{E|u(i)|* : t e [0,1]} < oo. 

Our optimal control problem is to find an admissible control it(-) minimizing 
the cost functional 

J(u(-)) = E [ l(t, x(t), u(t)) dt + Eh(x(l)). 
Jo 

In the above statement, A is the infinitesimal generator of a Co-semigroup, and 

/ : ft x [0, 1] x H x U ->• H, g : fl x [0, 1] x H x U -)• l 2 (H), 
I : fl x [0,1] x H x U ->R, h:flxH^R, 

where the Hilbert space 

f(H) := [z = ( Zi ;ie N) : \\z\\% {H) := \\z % f H < co}. 

Throughout this paper we make the following assumptions. 

Assumption 1.1 The operator A : D{A) C H — >• H is the infinitesimal gener- 
ator of a Co-semigroup {e tA £ B(H); t > 0}. Set 

M A := S up{\\e tA \\ B(H) :ie [0,1]}. 

Assumption 1.2 The functions f , g and I are allV xB(H)x B(U)- measurable, 
and h is J-\ x B{H) -measurable; for each (t,u,u) £ [0,1] x U x Q, f,g,l and 
h are globally twice Frechet differ entiable with respect to x; f x , g x , f xx , g xx ,l xx 
and h xx are bounded by a constant Mq; f 7 g,l x and h x are bounded by Mq{1 + 
\\x\\h + I and h is bounded by Mq(1 + \\x\\jj + 

In view of Assumption 11.11 the precise meaning of the equation (jl.ip is 

x(t) = c tA x + [ e (t - s)A [f(s 1 x{s),u{s))ds + g(s 1 x{s),u{s))-dW s ]. 
Jo 

A process x(-) satisfying the above equality is usually called a mild solution to 
equation (jl.lj) . cf. [3]. 

1.2 Developments of stochastic maximum principle and 
contributions of the paper 

The aim of this work is to find a stochastic maximum principle (SMP for short) 
for the optimal control. As we know, SMP is one of the basic tools to study 
optimal stochastic control problems. Since [llj . there have been a number of 
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results on such a subject. For finite dimensional system^, the problem in the 
general case was solved by Peng [13]. Hereafter, by the word "general" we 
mean the allowance of the control into the diffusion term and the nonconvexity 
of control domains. In contrast, most existing results on infinite dimensional 
systems are limited to the case in which the control domain is convex or the 
diffusion does not depend on the control (cf. [TJIHJIII])- Recently, several works 
p~2l [3 [5] were devoted to the general SMP in infinite dimensions. Lii-Zhang 
[T2"] first addressed such a problem and formulated a general SMP in which they 
assumed the existence of the second-order adjoint process. Fuhrman ct al. [7 
focused on a concrete equation (which was a stochastic parabolic PDE with 
deterministic coefficients) and gave a complete formulation of SMP, while their 
approach depended on the special structure of the equation. In our previous 
work [5] , a general SMP was obtained for abstract stochastic parabolic equations 
driven by finite Wiener processes. As far as we know, the general SMP for 
stochastic evolution equations as form (|l.ip is not completely solved. 

Our basic approach to derive the general SMP follows Peng's idea of second- 
order expansion in calculating the variation of the cost functional caused by 
the spike variation. The key point is how to understand and solve a B(H)- 
valued backward stochastic differential equation (BSDE) which characterizes 
the second-order adjoint process in our SMP. To do this, we exploit a concept of 
solution to this equation called "generalized solution" which was first proposed 
by Guatteri-Tessitore [5] in the study of infinite dimensional LQ problems, and 
prove the existence-uniqueness result in our framework. However, the general- 
ized solution only characterizes the first unknown component but says nothing 
about the second one. As a consequence, it seems difficult to derive our SMP via 
the traditional approach (i.e. applying Ito's formula). To avoid this difficulty, 
we first derive a basic property of the generalized solution. Then our goal is 
achieved thanks to the Lebesgue differentiation theorem. 

The rest of this paper is organized as follows. Section 2 is devoted to some 
preliminary results on SEE and backward SEE (BSEE) in the Hilbert space. 
In Section 3, we study the well-posedness and basic properties of an operator- 
valued BSDE. With the previous preparations, we shall state and prove our 
main theorem, the stochastic maximum principle, in the final section. 

We finish the introduction with some notations. Let H be a separable Hilbert 
space and £>(if) be its Borel a- field. The following classes of processes will be 
used in this article. Here p,q G [1, oo]. 

• Lp(f2 x [0, 1]; if) denotes the space of equivalence classes of processes x(-), 
admitting a predictable version such that E f Q \\x(t)\\ p H dt < oo. 

• Cp([0, 1]; L p (ri; if)) denotes the space of if -valued processes x(-) such that 
x(-) : [0, 1] — > L p (fl; if) is continuous and has a predictable modification. 

Moreover, since the a-field generated by the operator norm in B(H) is too 
large, we shall define the following spaces with respect to B{H )-valued processes 

1 For more related studies, we refer to 1161 and the references therein. 

2 Normally, the solution of a BSDE consists of a pair of adapted processes of which the 
second one is the diffusion term. For more aspects on BSDE, we refer to [gj. 
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and random variables. 

• L v v s (f2 x [0, 1]; B(H)) denotes the space of equivalence classes of B(H)- 
valued processes T(-) such that T(-)x £ L^,{VL x [0, l];i?) for each x £ H. Here 
the subscript "S" stands for "strongly measurable" . 

• Ljr t s (il;B(H)) denotes the space of equivalence classes of i?(-ff)-valued 
random variable T such that Tx £ £^(0; H) for each x £ H . 



2 Preliminary results on SEEs and BSEEs 

Let b : fl x [0,T] x H H and a : Ox [Q,T] x H -)• l 2 (H) be two VxB(H)- 
measurable mappings and F : O x [0, T] x H x / 2 (i?) -> be a P x #(-£0 x 
£>(^ 2 (i/))-measurable mapping such that 

\\b(t : x)-b(t,x)\\ H +h{t,x)-a{t,x)\\ l 2 {H) < M^x-xWh, 
\\F(t,x,y) - F(t,x,y)\\„ < Ah (\\x - x\\„ + \\y - y\\ lHH )) (a.s.) 

for some constant Mi > and any t £ [0, 1], x, x £ H and y,y £ l 2 (H). 

For given operator A satisfying Assumption 1 consider the following SEE 

x(t) = e tA x + f e^- s)A [b(s,x(s))ds + a(s,x(s)) ■ dW s ] (2.1) 
Jo 

and BSEE 

p(t) = e^-^'C + / e( s -*) A * [F( fl ,p(s), 9(a)) ds - • d^ s ] . (2.2) 



Now we present some preliminary results on SEE (|2 . 1 [) and BSEE (|2.2I) which 
will be often used in this paper. 

Lemma 2.1 Under the above setting, we have the following assertions: 

(l)Ifpe[2,oo),b(;0)£L^(nx[0,l];H) and a(>, 0) £ L p v (fix [0, 1]; l 2 (H)), 
then SEE (|2.1I) has a unique solution x(-) in the space C-p([0, 1]; L P (Q; H)) for 
any given xq £ H , with the L p -estimate 



E sup \\x(t)\\ p H < K(M A ,M 1 ,p)E 
te[o,i] 



.P/2- 

T(/.0)||? a(ir) dt 

(%) If F(-, 0,0) £ L 2 v (Vl x [0,1]; iJ), i/ien BSEE (gUJ ftos a imigue solution 
(p(")> ?(*)) * n ^ e 5 P ace 

Cp([0, 1]; L 2 (0; F)) x L 2 ,^ x [0, 1]; Z 2 (tf)) 
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for any given £ S Ljr^fJ; H), with the estimate 

E sup ||p(i)||^ + E / ||g(t)||? 2( mdt 
te[o,i] " Jo 

& + f ||F(t,0,0)||ird* 
Jo 



< K(Ma, Mi) E 



Hereafter K(-) is a positive constant depending only on the values in the 
brackets. 

The above results can be found in, for example, [3] [TO]. The following lemma, 
concerning the duality between SEE (|2.1[) and BSEE (|2.2[) . can be easily derived 
by the Yoshida approximation (cf. [TO]). 

Lemma 2.2 Under the conditions in Lemma \2.1[ we have 

ds 



E(x(fi),p(ti)) i3 - +E / (6(s,a;(s)),p(s)) ir + (o-(s,a;(s)),g(s)); 2(H) 
= E (x(t 2 ),p(t 2 )) H + E / 2 (z(s), F(s,p(s),q(s))) H ds 



for any < t\ < t 2 < 1. 

3 Operator- valued BSDEs: well-posedness and 
properties 

In this section, we study the following operator-valued BSDE (OBSDE) 

dP(t) = - {A*P(t) + P{t)A + A;(t)P(t) + P{t)A t (t) 

+ Tr[C*PC + QC + C*Q](t) + G(t)}dt + Q(t) ■ dW t , 
P(l) =P 1 (3.1) 

with the unknown processes P(-) and Q(-), where A satisfies Assumption II. H 
A$(-) and C(-) are given coefficients, and 

Tt[C*PC + QC + C*Q](t) = y [CfPd + Qid + C*Qi]{t). 

We call the pair (G, Pi) the input of OBSDE p. II) . Such a equation will be 
used to characterize the second order adjoint process of the controlled system 
in the next section. We make the following assumption. 

Assumption 3.1 A$(-) G £p S (^ x [0, 1]; B(H)). C(-) = (Ci(-) ; i g N) with 
Ci(-) S L™ s (tt x [0,1]; B(H)).' Assume 



M 2 := esssup i)W {||A s (i,u;)||| (H) , \\d(t, u) \\ 2 B(H) } 



< oo. 
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3.1 The well-posedness 

The solvability theory of £?(iJ)-valued BSDEs is still far from complete. A first 
remarkable work on such a subject was found in Guatteri-Tessitore [5] where, 
inspired by the notion of "strong solution" for PDEs (cf. [2]), the authors 
proposed for an OBSDE the concept of generalized solution which only involved 
the first unknown P(-), and obtained the corresponding existence-uniqueness 
result. Their approach based on the solvability of (|3.1[) in the Hilbert space 
B2{H) of all Hilbert-Schmidt operators from H to itself, see Theorem 5.4 in [5]. 
Following the spirit of Guatteri-Tessitore [8] , we give the following definition. 

Definition 3.2 (generalized solution) P(-) is called a generalized solution 
to OBSDE (HU) inL^ s (nx[0,l];B(H)) if there exists a sequence (P n ,Q n ,G n ) 
such that 

1) P n {l) G L 2 J r i {fl:B 2 {H)), G n G L 2 v {n x [0, 1]; B 2 (H)), and there exists a 
constant A > 1 such that 

\\P n (l)\\B(H) < M\Pi\\b(h) and \\G n (t)\\ B(H) < X\\G(t)\\ B{H) (a.s.) 

for any neN and t G [0, 1] . 

2) (P n , Q n ) is a mild solution^ to OBSDE (J3T]) with the input (G n , F"(l)) } 
that is, for all t G [0, 1], 

P n (t) =e (1 ^ A *P n (l)e (1 -^ A + £ e^ s - t)A * [A;P n + P n A t + G n }(s)e^ t)A ds 
+ J e( s -Q A *Tr[C*P n C + Q n C + C*Q n }(s)e (s -^ A ds 
+ j\ {s - t)A *Q n (s)e( s ~^ A -dW s , (a.s.). (3.2) 

3) for any x,y G H and t G [0, 1], 

(x,G n (t)y) H ^(x,G(t)y} H and (x,P n (t)y) H -> (x,P(t)y)„ (a.s.) 

In the above definition only the process P(-) is characterized. Nevertheless, 
this is sufficient and even natural for the optimal control theory since Q(-) is not 
involved in the formulation of the SMP (see Theorem 14. 1[) . For more detailed 
account we refer to Remark 6.3 in [8]. Now we give the following well-posedness 
result on the generalized solution to OBSDE (|3.ip . 

Theorem 3.3 Let Assumvtions \ 1 . l\ and [37l\ be satisfied. Suppose Pi G Lj? s (ft; B(H)) 
and G G x [0, 1]; B{H)). Then 

i) there exists a unique generalized solution P(-) to OBSDE (|3.1[) .- 

3 See Definition 5.3 in [8]. 
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ii) for each t G [0, 1] and any £, £ G Ljr (ft; i?), 
(^P(t)Oh = E^^l),^^!)^ 



+ E 



(^(t),Gf(t)y T ^(t)> H <« (a.s. J (3-3) 



wii/i j/ T '^ (similarly for y T ^ ) being the solution to equation 



iii) for each r G [0, 1], it holds almost surely that 



P(r)f B(H) <K(M A ,M 2 



7? r 



\Pl\ 



B(H) 



\\G{t)\\ 



B{H, 



At 



(3.4) 



A T ; (3.5) 



iv) for each r G [0, 1) and any £, C G P^- (ft; if), 



lim ] 

s-il-s-0 



; <£, (P(a) - P(t))C) H = with s, t G [r, 1]. 



The proof of the above theorem depends on the following result in which the 
OBSDE (|3.1j) is considered in the Hilbert space B 2 {H). Such a lemma is similar 
to Theorems 5.4 and 5.5 in Guatteri-Tessitore [5]. 



Lemma 3.4 In addition to the conditions in Theorem \3.SX suppose P\ G P^ 7 (^i ^2 
and G G P?>(ft x [0, 1]; B2(H)). Then there exists a unique mild solution 
(P(-),Q(-)) in the space 

C V {[Q,1]-L 2 {n-B 2 {H))) x L 2 v (n x [0,1]; l 2 (B 2 {H))), 

that is, {P,Q) satisfies relation p. 21) instead of (P n ,Q n ). Moreover, the asser- 
tions (ii) and (iii) in Theorem \3. 3\ hold true with respect to such P(-)- 

Proof. The existence and uniqueness of the (mild) solution (P(-),Q(-)) were 
given by Theorem 5.4 in [8]. Next, we indicate that Lemma [2~T1 implies 



y T ^(r)f H <K(M A ,M 2 ) MWl (a.s.) 



(3.6) 



for any r G [0, 1] and £ G (ft; H). In the case of A G B(H), one can show 
the relation (|3.3p by the generalized Ito formula (see e.g. [31 Theorem 4.17]). 
Then the general case can be obtained by a standard argument of the Yoshida 
approximation. The inequality (I3.5[) follows from (|3.3p and 



Proof of Theorem 13.31 For the sake of convenience, we write the right-hand 
side of equality p.3[) as T T (£, £; G, Pi). Then it follows from (|3.6|) and Young's 
inequality that for any £, £ G Pjr x (ft; if), 



|T T (e,C;G,a)|< v^lKIUCllff (a.s.). 
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We shall always select an RCLL version of the martingale (A T ; r G [0, 1]). Fix 
arbitrary r G [0, 1] and take a standard complete orthonormal basis \ef } in H. 
Then there is a set of full probability fix C £1 such that for each uj G fii, 

|[T T (ef,ef;G,i\)] (w)| < v/A^R, Vt,j G N. 

Hence, from the Riesz representation theorem, there is a unique P(r, cj) G B{H) 
for each w G fii such that 

(ef ,P(T,w)ef ) fl - [T T (ef ,ef ;G,Pi)] (w), Vi.j G N, (3.7) 

and 

||-P(T,w)||fl(fl) < V A rM, VwGfii. 

It is easy to check that (x,P(t)u)h = [T T (x,y;G, Pi)] (a.s.) for any x,y ^ H: 
and furthermore, for any simple iJ-valued Jv-measurable random variables £, £, 
we have 

(e,P(r)C)H = [Tr(e,C;G,P 1 )], (a.s.) 

Then by a standard argument of approximation, the above relation holds true 
for any £,( G L% T (Cl;H). 

We claim: the operator-valued process P(-) defined in (13. 7[) is the desired 
generalized solution. Indeed, for each n G N, we introduce the finite dimensional 
projection LI„ : H — > iJ : x — > X)"=i( x ' e f)H e f , define 

G n (t,w) := n„G(i,w)n„ and P"H := n n P x (o;)n„ 

and find from Lemma E3] the (unique) mild solution {P n ,Q n ) to OBSDE (|3~T]) 
with the input (G n ,P"). Obviously, the conditions (1) and (2) in Defini- 
tion 13.21 are satisfied. Noticing the construction of G™, it remains to show 
(x : P n (t)y) H -t (x,P(t)y) H (a.s.) for any x,y G H and t G [0,1]. It follows 
from Lemma GOD that (x,P n (T)y) H = T T (x,y 1 G n ,PJ l ) (a.s.). By the Lebesgue 
dominated convergence theorem, we have 

T T {x, V] G n ,P?) -+T T {x i y;G,P 1 ) (a.s.). 

This implies {x,P n (T)y)H — > {%, P{t)v)h ■ The claim is proved. 

On the other hand, from Lemma 13.41 Definition 13.21 and the Lebesgue dom- 
inated convergence theorem, the relations (|3.3j) and (|3.5|) hold true for every 
generalized solution to OBSDE (|3.1j) . which yields assertions (ii) and (iii). More- 
over, the relation (|3.3j) implies the uniqueness of the generalized solution. Thus 
the assertion (i) is proved. 

It remains to prove the assertion (iv). Fix arbitrary r G [0, 1). Without loss 
of generality, we assume t < s. Then for any £, £ G (£l;H) we have (recall 
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equation (|3.4[) ) 

E(£, (P(s) - P(t))Q H 

+ E / [(y s >t(r),G(r)y s >t(r)) H - (y^(r),G(r)y^(r)) H ]dr 



-EJ (y^(r),G(r)y^(r)) H dr 

=: h+h+ h 
First, it follows from (13.61) and Young's inequality that 

\h\ 2 < K(M A , M 2 )eJ° \\G(r)\\ 2 BW dr ■ tJe \\^\\% ■ E \\C\\% -> 0, (3.8) 

as \s — t\ — y 0. On the other hand, the trajectory of y T '^(-) is continuous in H, 
which along with p.6[) and the Lebesgue dominated convergence theorem yields 



\h\ 2 < K(M A , M 2 , G, Pi)(^E U\\h\/^\\C - y^i^C 

+ y/vK\\ 4 H\/m-y t Hs)t H )^0, as |* — *I 0. 
Similarly, we can show 

|Zx| — >- 0, as |s-t| ->•(). 
Therefore, we have for any f, £ G (fi; ff), 

lim E (£, (P(s) - P(t))Q H = 0, with s,t G [r, 1]. 

|s— 1|— »0 

The assertion (iv) is proved. This concludes the theorem. ■ 

3.2 A basic property of the generalized solution 

The absence of Q(-) in the definition of generalized solution brings a new diffi- 
culty in our derivation of the stochastic maximum principle compared with the 
traditional duality approach (cf. 13 ). The following result will play a key role 
to overcome the difficulty. 

Proposition 3.5 Let the conditions in Theorem \3.S\ be satisfied. For any t G 
[0, 1), 9 G L a - Ft (fi; l 2 (H)) and e G (0, 1 - t), let yl>*{-) be the mild solution to 
equation 

V T /{t) = f t e^ A A i (s)y^(s)d S 



[ T ^~ S)A [c(s)y T ^(s) + e-h [T , T+e) # 



■dW». 
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Then we have 

E(#,P(t)6) iHh) 



(y^(l),P W l' e (l)) H + / (y^{t),G{t)yl'\t)) H At 



= limE 

Remark 3.6 In the B2(H) -framework, the above result can be easily proved 
by the ltd formula ( recalling Lemma \ 3.4]) ; however, this approach fails in the 
general B(H) -framework due to the absence of Q(-) in the generalized solution. 
Besides, an approximation argument by using the sequence (P n ,Q n ,G n ) seems 
also difficult to prove the above result. 

The previous proposition follows from several lemmas. For the sake of con- 
venience, we denote 



T^,6) =E(y^(l),P iy ^(l)> K + E J (y^(t),G{t)yl' e (t)) H dt. 
Define 

B A (r) :={ee4 T (fi;ff):e( U )efl(i) 

and ||e|U:= S up(||^|| H + ||eM<oo} 

which is dense in Ljr (f2; H). Set = (0, . . . , 0, 1, 0, . . . ) with only the i-th 
element nonzero. Then £a G Ljr T (fl;l 2 (H)) for any £ <E L^ t (Q;H) and i G N. 
Moreover, we define 

f (i) := e-^Wi - W*)Z and ? (t) := e -*(W/ - W})(. 

Lemma 3.7 For any t G [0, 1), £ G Ba(t) and i 6 N, we Ziawe 

E||y^(r + £ )-r(r + £)|| 4 ff <^ 2 ||£||^. 

Proof. For simplicity, we set y 1 ^) := yl>^ ei (t). Then for t G [r, r + e], 



t 



fo* -£*)(*) = / e(*-^[^( S )(^-f)( S ) + (^ + ^( S ))f( S )]d S 
^ A [C(s)(f - £«)(«) + C(s)C(s)) ■ dW t . 
Then by Lemma 12. 11 we have 

E || (J/ 1 - e*)(r + < KE( ||f dt) * < Ke 2 . 
The lemma is proved. ■ 
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Notice the fact that for any £, £ £ Ba{t), 

+ E (y^ (r + s), P(t + e)y^ (r + e)) H 
=■ Ji + J 2 - 

Now we let e tend to 0. On the one hand, one can show that the term J\ tends 
to similarly as in p. 81) ; on the other hand, by means of Lemma 13.71 the term 
J2 should tend to the same limit as E(C (r + e), P(t + e)( 3 (t + e)) H . Indeed, 
we have 

Lemma 3.8 For any r £ [0, 1), £, ( £ Ba{t) and i,j £ N, we have 
hm {E (f(r + e),P(r + e)C\r + e)) H - T T ^e 4 , Qe 3 )} = 0. 

Proof. It is sufficient to show 

hm {E (C (r + e), P(r + e)C (r + e)) H - J 2 } = 0. 

Indeed, from Theorem 13.31 Lemmas 12.11 and 13. 71 we have 

\E(C(r + e) 7 P(T + e)C(r + e)) H -J 2 \ 

< K (EU\\%) 1 (E\\y^ (r + e)- ? (r + e)\\%) ' 

+ X (E||C|| 4 ff ) 1 (E||yJ^ (r + £ ) - f (r + E ) || 4 ff ) 4 
— > 0, as e J, 0. 

This concludes this lemma. ■ 

On the other hand, from the continuity of P(-) we have the following 

Lemma 3.9 For any t £ [0, 1), £, £ G Pa(t) emrf G N, we ftawe 

hmE (f (r + e), P(r + e)^ (r + e))^ = E <£, P(r)C) H . 

Proof. It is easy to see 

E (f (r + e), P(r)C J (r + e)) H = E (£, P(t)() h . 
Thus we need prove 

hm E (f(r + e), [P(r + e) - P(t)]C(t +e)) H = 0. 

It follows from (|3.5|) . the boundedness of £, £, and Doob's martingale in- 
equality (cf. 01) that (recall (|33]l ) 

|<£, [P(r + e) - P(r)] C) H \ 2 < 4(max 4e[0!l] A t ) ||£[[ 2 H \\C\\ 2 H £ L 1 ^), 
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Note that 

(e(r + e),[P(r + e)-P(r)}CHr + e)) H 

= e-\W\ +E - W*)(W} +e - Wl) (£, [P(r + e) - P(r)] Q H 

Then from Theorem 13. 3f iv) and the Lebesgue dominated convergence theorem, 
we have 

\E(C(r + s), [P(r + e) - P{t)]?{t + e)) H \ 2 

< e- 2 ■ E(m +e - W*\ 2 \W? +e - W?\ 2 ) • E|<£, [P(r + e) - P(r)} () H | 2 

< 3E |(£, [P{t + e) - P(t)] C) h \ 2 -> 0, as e | 0. 

The lemma is proved. ■ 

Now we are in a position to complete the proof of Proposition 13.51 
Proof of Proposition dm Fix any ■&, 6 G 1^(0; Z 2 (-H")). For arbitrary S > 0, 
we can find (from the density) a large m<5 and £i ; i = 1, . . . , mj} C P>a(j) 
such that 

• — (£l j • • • j frn 5 ): @rn$ - = (Cl ) • • • ) Cm,5 ) j 

E - # ms \\p {H) + E ||0 - 7? m J|f 2 < <5 4 . 

Then we have 

|E (0, i'WJJptjr) - E (0 m „ P(r)& m3 )p W | < K{0, 9, P) 6, 
\T^t},6)-TC(i3 ms ,9 ms )\ < K(V,6,P)8. 
On the other hand, from Lemmas 13.81 and 13.91 one can easily check that 



E(# ms ,P(T)6 ms ) l2(H) = limT^ ms ,d ms ). 



Thus we have 



limsup|E(tf,P(T)6l) i2(H) -T T £ (tf,(9)| < K{d,9,P)5. 

E-10 

From the arbitrariness of 5, we conclude the proposition. ■ 

4 The stochastic maximum principle and its proof 

4.1 The statement of the main theorem 

Now we are in a position to formulate the stochastic maximum principle for 
optimal controls. Define the Hamiltonian 

H: tt x [0, 1] x H xU x H x 1 2 {H) -> R, 

as the form 

U(t,x,u,p,q) := l(t,x,u) + (p, f(t, x,u)) H + (q,g(t,x,u)) l2(H) , (4.1) 
then our main result can be stated as follows. 
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Theorem 4.1 (stochastic maximum principle) Let Assumptions \1.1\ and 
II. £1 be satisfied, x(-) be the state process with respect to an optimal control u(-). 
Then for each u 6 U , the variational inequality 

<H(T,x(T),u,p(T),q(T)) -n(T,x(T),u(T),p{T),q{r)) 

+ 2 (9(t,x(t),u) - g(T,x(T),u(T)),P(T)[g(T,x(t),u) - g(r, x(t), u(t))]) P(h) 

holds for a.e. (t,u>) £ [0, 1) x Q, where (p(-), ?(•)) is the solution to BSEE (12.21) 
with 

F(t,P,q) = H x (t,x(t),u(t),p,q) , £ = h x (x(l)), 
and P(-) is the generalized solution to OBSDE (|3.1[) wraf/i 

A|((t) = /*(i,ic(t),u(t)), C(t)=g x (t,x{t),u{t)), 
G{t) = H xx (t,x(t),u(t),p(t),q(t)), P l = h xx (x(l)). 

4.2 Proof of the main theorem 

The proof is divided into the following three steps. 

Step 1. The spike variation and second- order expansion. 

The approach in this step is quite standard (cf. [13]). Recall that x(-) is the 
state process with respect to a optimal control u(-). We construct a perturbed 
admissible control in the following way 



u e (t) 



u, iite [t, t + s], 
u(t), otherwise, 



with fixed r £ [0,1), sufficiently small positive e, and an arbitrary [/-valued 
J>-measurable random variable u satisfying E|u||y < oo. 

Let x e {-) be the state process with respect to control u £ (-). For the sake of 
convenience, we denote for ip = f, g, I, f x , 

9x i t-x ) fxx : 9xx ■) ^XX ) 

Cp{t) ;= ip(t,x(t),u(t)), 
^ A {t) := <p(t, x(t), u £ {t)) - <p(t, x(t),u(t)), 

Let X\(-) and x 2 (-) be the solutions respectively to 
xi(t)= f e^ A f x (s)x 1 (s)d S + f e^ A [g x (s)x 1 (s)+g A (s)}-dW s , 

ds 

1 -dW s . 



JO 

1 - ,-A, 



x 2 (t) = / e^ A f x (s)x 2 (s) + |/ Xic (s) (xi ® xi) (s) + 



1 _ 

g x (s)x 2 (s) + -g xx (s) (xi ®xi) (s) + g x (s)xi(s 
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It follows from Lemma I2TT1 that for all t € [0,1], 



f e- 2 E \\ Xl (t)\\% + e^E \\ Xl (t)f H + e~ 2 E \\x 2 (t)\\ 2 H < K, 
e- 2 E\\x E {t) - x(t)\\% + e^EWx" (t) - x(t)\\ 2 H < K, 



(4.2) 



£- 2 E\\x £ (t) - x(t) - Xl (t)\\ 2 H < K, 
I e" 2 E ||a; £ (t) - x(t) - n(t) - sb 2 (*)||^ = o(l). 
This along with the fact 

J(u £ (-))- W))>o 

yields (for details, we refer to [TB] or [5]) 

/•l r I , 

o(e) <E/ i A (t) + + a;2(t)) ff + - W, L(t)n(t)) H dt 

+ E (h x (x{l)), Xl (l) + x 2 {l)) H + 1 (xi(l), ^(^(lJJaiiCl))^ . 
S'iep First-order duality analysis. 

It follows from Lemma l2~l7 2) that BSEE (|2.2|l has a unique solution (p(-), 9(0) 
in this situation. Recalling the Hamiltonian (|4.1[) . and from Lemma 1^21 we have 

[; A (i) + (f a (t), xx{t) + x 2 (t)) H ] dt + E (h x (x(l)), Xl (l) + x 2 {l)) H 

rT+s 

o{e)+E I [U(t,x(t),u,p(t),q(t))-H(t,x(t),u(t),p(t),q(t))]dt 
1, 



-E 



l . 



(p(t),fxx(t) (xi + (q(t),g xx (t) (xi ®xi) (t)) l2{H) 



dt. 



Hence, we get 



r+e 



o(l) < e _1 E / [H(t,x(t),u,p(t),q(t)) -n(t,x(t),u(t),p(t),q(t))] dt 



+ 2 £_1]E / (^iW,^x(t,5W,S(i),p(t),g(0)^iW)H dt 



+ -e- i E(x 1 (l),/ l;c;c (5(l)) a;i (l)) H . 



(4.3) 



>f)tep 3. Second- order duality analysis and completion of the proof. 
This is the key step in the proof. From Theorem 13.31 it is easy to check that 
OBSDE (|3.1j) has a unique generalized solution P(-) in this situation. Now we 
introduce the following equation 
rt 



y E (t) 



> A A t (s)y e (s)ds 



+ e^ A [C(s)y £ ( S )+e-h^ T+E) g A (r)}- dW s , te[r,l}. 



Then we have 
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Lemma 4.2 For a.e. t € [0, 1), we have 

lim sup E\\e~^xi(t)-y e (t)\\ 4 =0. 

te[r,l] 

Proof. By Lemma [23] we have for each r 6 [0, 1], 

sup E||e-*a;i(*)-i/ e (t)||t r < K • - / + E \\g A (t) - g A (r)||' At. 
te[r,i] " £ Jt y ' 

From the Lebesgue differentiation theorem, we have for each X € (f2; l 2 (H)), 

1 f T+C ,, A ,,4 M A ,,4 

lim-y E||</ A (t) -X\\ l2{H) dt = E|| 3 a (t) - X\\ p[H) , for a.e. r e [0,1). 

Since Lj-^fi; l 2 (H)) is separable, let X run through a countable density subset 
Q in 1^(0; l 2 (H)), and denote 

i? := I^J .Ex :— (J {r : the above relation does not hold for X }. 

Then we have meas(-E') = 0. For arbitrary positive 77, take anleQ such that 

E\\g A (r) - X\\ 4 l2(H) <r), 

then for each t € [0, 1)\-E, 

1 / tt-. 1 1 A / .\ A / \ 1 1 4 



lim^ / E|| ff A (t)- 3 A (r)||; dt 



<lim- / E|| 5 A (t) — JSCll* dt + 8E|U A (r) -X| 

<16E||.g A (r)-X||" 2(H) <16t7. 

From the arbitrariness of 77, we conclude this lemma. ■ 
From the the above lemma, we have 



4 

P(H) 



e^E / <a:i(t),G(i)a;i(i)) H dt + s^E ( Xl (l), i\» 1 (l)) H 
Jo 

= o(l) + E / (y E (i), G(t)y £ (t)) H dt + E (y E (l), Pi?/(l)) ff , Vr e [0, 1)\E. 



Keeping in mind the above relation, and applying Proposition ^. 5[ we conclude 
for each r G [0, 1)\E, 

E(g A (r),P(r)g A (r)) l2{H) 

= time' 1 <E (x 1 (t),G(t)x 1 (t)) H dt + E{x 1 (l),P 1 x 1 (l)) H 
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This along with (|4.3[) and the Lebesgue differentiation theorem yields for each 
u € U, 

< E[n(r,x(T),u,p{T),q(r))-n(T,x{T),u(T), P (T),q(r))} 



1 



+ -E{g A (T),P(r)g A (T)) l2{Hr a.e. re [0,1). 

Therefore, the desired variational inequality follows from a standard argument 
(cf. |llj). This completes the proof of the stochastic maximum principle. 
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